Hi Boston!

• I’m Jasmine Dumas
  • Data Scientist and useR living in Connecticut
  • Work at Simple Finance based in Portland, OR
  • Author of ttbbeer and shinyLP
• Slides and Materials available on GitHub:
  • jasdumas/talks/odsc-boston

Why should you care open government data?

To foster and improve data literacy & statistical comprehension within our community, Scientists and Engineers need to advocate for data that is provided in an consistent and accessible method suitable for analysis.

What this presentation is about (and not about)

• The state of open government data in relation to beer analytics and the difficulties in finding and exploring analysis-ready data in a never ending sea of data
• This is not about “drunk politics” or regulations, hacking, or scandals regarding data

2017 has been an interesting time to be a Data Scientist…

https://www.nytimes.com/2017/03/06/science/donald-trump-data-rescue-science.html?smid=tw-nytimesscience&smtyp=cur&_r=0

What even is, beer analytics?

• The use of quantitative methods to:
  • improve the production & quality of beer
  • measure customer preferences and predict seasonality
  • gather insights on industry performance

Discovering analysis-ready beer datasets can be difficult

• Datasets: Craft Beer Sample Data, SNAP-RateBeer* | Accessible to academic researchers only
• API’s: RateBeer, Untapped, The Beer Mapping Project, BreweryDB | Restrictions on usage or require a user account
• Projects about beer: yhat Beer Advocate Recommender and Crowd-sourced Beer Distribution Map

How to search for analysis-ready datasets in general

• Google public data explorer
• Github
• Kaggle data repository
• Machine Learning Repositories at Universities: UCI
• Open Government Data Portals: Boston, Connecticut, SF, and many more

The U.S. Government Open Data Portal

The “clearinghouse” for open U.S. government data is located at data.gov. It also contains tools & resources to conduct research, develop web and mobile applications, and design data visualizations.

Examples of analysis-ready datasets

• Analysis-ready datasets can be obtained with R functions like:
  • read.csv() or read.table()
• “Great” analysis-ready datasets take minimal cleaning and have interpretable data fields
• Analysis-ready data also needs to include some type of documentation such as a data dictionary

Examples of datasets that are not analysis-ready

• Datasets that have inconsistent formats, mixed column types, and cryptic data fields make it difficult for analysis

Here is the point of all of this

Just because it’s open doesn’t mean it’s accessible!

Necessity breeds innovation

• I was inspired to become better at finding datasets and web scraping from the road-blocks of finding analysis-ready datasets for projects about beer!

• Here is a tutorial I wrote to get meta data on brewery ratings in Connecticut from Beer Advocate using Import.io and processed the data with R.

I developed a R package for beer statistics, called ttbbeer

• This R data package provides one dataset for materials used at U.S. breweries as listed in the Beer Monthly Statistical Releases, and eight datasets for historical tax rates of distilled spirits, wine, beer, champagne, and tobacco from the U.S. Department of the Treasury, Alcohol and Tobacco Tax and Trade Bureau (TTB)

• Available on CRAN and Github

How I developed the ttbbeer package

• Hand transcribing from PDF documents to an excel workbook 😭
• Web-scraping using the R package rvest 😀

Insights from this data

Advocating for open data

https://www.data.gov/issues/request-id/32022/

Questions & Discussion!